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Abstract Body 

Background / Context: Community colleges play a vital role in American postsecondary 
education. They enroll nearly half of all college students nationwide and, because of their open 
admissions policies and low cost relative to most four-year institutions, are accessible to millions 
of adults who might otherwise lack the preparation or financial means to pursue higher 
education. Unfortunately, this open access does not always translate into academic success. A 
national study found that only a third of students who began at community colleges had obtained 
a degree within six years (Berkner, He, & Cataldi, 2002). Many community college students 
want to earn a degree, but are stymied by the competing demands of work, family, and school. 
Institutional barriers, such as poorly tailored instruction, insufficient financial aid, or inadequate 
counseling, can also impede students’ academic progress (Brock, LeBlanc, & MacGregor, 2005). 
As a result many community colleges are searching for innovative programs to improve the 
likelihood of academic success among their students. 

Many reforms have been implemented to increase rates of college persistence and 
completion for disadvantaged students. In the past decade, some have been studied in a way that 
examines the causal relationship between the reforms and students’ educational progress and 
attainment — using a random assignment design. Most of the higher education initiatives studied 
rigorously fall into three categories — financial aid reforms, enhancements to student services, 
and instructional reforms. A variety of approaches have been found to help improve students’ 
educational outcomes (for examples, see: Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2009; 
Goldrick-Rab, Harris, Benson, & Kelchen, 2011; Richburg-Hayes et ah, 2009; Richburg-Hayes, 
Visher, & Bloom, 2008; Scrivener & Weiss, 2009). Overall, the rigorous research has shown 
that, encouragingly, something can be done to improve success rates for postsecondary students. 
However, the evidence to date suggests that single-focus, light-touch interventions may not be 
robust enough to substantially improve outcomes, and short-term enhancements are likely to lead 
to only short-term gains (for example, see: Scrivener & Coghlan, 2011). The growing body of 
research evidence points to the need for multi-faceted, longer-lasting interventions. This 
presentation will present three-year findings from a program designed with this more 
comprehensive, long-lasting approach in mind. 

The City University of New York’s (CUNY's) Accelerated Study in Associate Programs 
(ASAP) combines many of the ideas from a range of programs into a comprehensive model that 
requires students to attend school full-time, and provides supports and incentives for three years. 
ASAP’s financial aid refonns, enhanced student services, and scheduling reforms are designed to 
increase the proportion of students who graduate with an associate’s degree. 

Purpose / Objective / Research Question / Focus of Study: Broadly speaking, the primary 
objective of the study is to determine: 

• What is the effect of the ASAP program on students’ academic outcomes? 

This question examines whether ASAP is effective compared with the business-as-usual services 
offered at the participating colleges. This is the question at the heart of the evaluation. 

In addition, research questions focusing on the implementation of the ASAP program are 
considered — questions that are critical for understanding and interpreting the program’s effects. 
For the implementation analysis, the study is designed to address the following questions: 

• How is ASAP put into practical effect (that is, how is it implemented)? 
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Understanding how ASAP is implemented will provide important information for other colleges 
and college systems interested in implementing ASAP or a similar program. 

• How do the planned program services compare with the offered program services (that is, 
to what degree is there fidelity to the program model)? 

ASAP is an ambitious intervention, comprised of many components. It is anticipated that some 
of these services will be offered as they were conceived, whereas others may fall short of initial 
plans or be changed over time in order to better meet the needs of students. Understanding 
program fidelity is vital for interpreting the program’s effects. 

• How are the services received by program group students different from those services 
received by the control group members (to what degree is there a treatment contrast)? 

In an experimental evaluation it is absolutely critical to understand the treatment contrast, as it 
represents the likely cause(s) of any program impacts on student outcomes. If program group 
members visited academic counselors three times per semester and control group members 
visited academic counselors one time per semester, then these two additional visits may be 
partially responsible for observed program effects on outcomes such as credit accumulation and 
degree receipt. Should ASAP have positive effects, this infonnation may shed light on which 
program components made the difference (the ones with a treatment contrast) and which did not 
(those without a treatment contrast). 

• What does ASAP cost? What is ASAP’s net cost? Is ASAP cost-effective? 

ASAP’s services are substantially more expensive than business-as-usual. Estimating how much 
more expensive provides important context when interpreting this studies’ findings. 

Setting: CUNY, the site of this project, is the largest urban public universities in the country, 
comprising 24 colleges and other institutions, serving approximately one-half million students 
each year. In the fall of 2010, CUNY’s six community colleges enrolled a total of 91,264 
students, 65 percent of whom were Pell eligible, and 48 percent of whom were first generation 
college students (CUNY Office of Institutional Research and Assessment, 2011). Although most 
CUNY community college students start out by attending college full time, nearly half drop to 
part-time status at some point during their college careers, dramatically decreasing their chances 
of obtaining a postsecondary degree in a timely fashion (Lindennan & Kolenovic, 2009). In fact, 
only 24 percent of students who started as first-time, full-time freshmen in 2001 earned an 
associate’s degree within six years (CUNY Office of Institutional Research and Assessment, 
2008). This study includes the three largest of CUNY’s six community colleges. 

Population / Participants / Subjects: This study had several eligibility criteria - students had to 
be: low-income, in need of one or two developmental courses, a new or continuing student who 
had earned 12 or fewer credits, and willing to attend college full time. Table 1 presents selected 
characteristics of the 897 individuals in the research sample. The first column shows the data for 
the full sample, which provides a descriptive profile of the students in the study. 

Similar to community college students nationwide, the majority of study participants are 
women (62.0 percent). 39 percent of participants were between 16 and 18 years old when they 
enrolled in the study, and 26. 1 percent were 19 or 20 years old; that is, most sample members 
were of traditional college age. The study sample is racially diverse, with no racial majority — 
43.5 percent of sample members identified themselves as Hispanic, 10.0 percent as White, 34.4 
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percent as Black, and the rest as Asian, Pacific Islander, or other. Just over 60 percent of sample 
members were incoming freshmen at the start of the study. Nearly 45 percent spoke a language 
other than English at home, reflecting the diversity of the population of New York City. 

The second and third columns divide the sample into the program and control groups, 
allowing for the comparison of program and control group members’ background characteristics. 
As the table shows, random assignment resulted in research groups with similar sample members 
at the start of the study. 

Intervention / Program / Practice: The three-year ASAP program bundles together a group of 
strategies — financial aid reforms, enhanced student services, and schedule reforms — many of 
which have a research base on their effectiveness. ASAP both provides a wide array of services 
and supports to help students graduate and includes requirements and messages to students about 
their schooling. Most ASAP components are provided for three years (except where indicated 
below). It is notable that although ASAP provides a broad range of supports, it also requires and 
expects a lot from participating students. Below are the key components of the ASAP model — 
ASAP as designed. 

Requirements and Messages 

• Enroll full time. ASAP requires students to enroll full time (12 or more credits) in the fall 
and spring semesters. 

• Take developmental courses early. ASAP encourages students to take any developmental 
courses they need to early in their time in college. 

• Graduate quickly. ASAP encourages students to graduate with an associate’s degree within 
three years 

Course Enrollment 

• ASAP seminar. Students are required to take a non-credit ASAP seminar that covers topics 
such as goal-setting and academic planning for at least their first year in the program. 

• Block-scheduled classes. Groups of students organized by major take three or more courses 
in a consolidated block schedule during their first year in ASAP. The block includes the 
ASAP seminar and typically a developmental education course. 

Student Services 

• Comprehensive advisement. Students are assigned to an ASAP adviser with a small 
caseload: 60-80 students. (Typical community college advisers can be responsible for 1,000 
students or more.) ASAP advisers are expected to provide comprehensive academic and 
interpersonal support. Students are required to meet with their adviser at least twice a month. 

• Tutoring. ASAP tutors provide general support both inside and outside the classroom. Some 
ASAP students are required to attend tutoring frequently, such as those taking developmental 
courses or those on academic probation. 

• Career services. ASAP career and employment specialists help with career planning, and, if 
needed, job placement. Students are required to meet with the career and employment 
specialist once a semester. 

Financial Supports 


Please note that ASAP does not directly model itself after the exact reforms that have been a research base on their 
effectiveness. Rather, the ASAP program borrowed ideas from these three areas. 
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• Tuition waiver. ASAP waives any gap between students’ financial aid and their tuition and 
fees. (ASAP requires students to complete a Free Application for Federal Student Aid) 

• Free MetroCards. The program provides free monthly MetroCards for use on public 
transportation. The MetroCard is tied to fulfilling certain program requirements, such as 
meeting with an adviser or attending tutoring. 

• Free textbooks. ASAP provides free textbooks for all classes. Students must return them at 
the end of the semester. 

Research Design: This study is a randomized field trial. Participating students were assigned, at 
random, either to the program group, whose members have the opportunity to participate in 
ASAP, or to the control group, whose members receive the college’s standard services. The 
random assignment procedure and process was designed and controlled by the researchers. 

Data Collection and Analysis: The following data sources are used in the analyses: 

• Baseline data : After students agreed to be a part of the study and prior to being randomly 
assigned, students filled out a Baseline Information Fonn (BIF) and baseline survey. 

• Field Research : The researchers are conducting periodic interviews with ASAP staff 
members, including directors, advisers, career and employment specialists, and tutors. 

• Student 12-month Survey : Covers topics including sample members’ (program and control) 
use of program services, and measures of mediating variables (e.g., reduced hours working 
and increased time studying). The response rate for the survey was around 80 percent. 

• Transcript data : These records include the following from all CUNY colleges: transcript data 
(e.g. course name, credits and grades); basic skills assessment test data (e.g. test type, score, 
date, and site for all attempts); and degree data (e.g. type, major, date, and college). This data 
was provided to us by the CUNY Institutional Research Database (IRDB). 

• National Student Clearinghouse : These records include enrollment and degree attainment 
from over 95 percent of the nation’s colleges. 

Findings / Results: The findings presented here represent two-year results, which have been 
presented at SREE in the past. By the time of the Fall SREE conference, three-year graduation 
impact estimates will be publicly available, as will results from the student survey, cost analyses 
and significant qualitative research. Within two years, students offered ASAP clearly 
outperfonned their control group counterparts with respect to persistence, credit accumulation, 
and graduation. Figures 1 and 2, and Table 2 summarize the findings. During each semester, 
ASAP students enrolled at a higher rate than their control group counterparts. After two years, 
students offered ASAP earned an average of 37.9 credits and their control group counterparts 
earned an average of 30.3. Thus, ASAP is estimated to have increased credit accumulation by 7.6 
credits. Two-year graduation rates were 8.7 percent for the control group and 14.5 percent for the 
program group; thus, ASAP is estimated to have increased graduation rates by 5.7 percentage 
points. Two and a half year graduation effect results are available for a partial sample and 
suggest dramatic effect estimates in the range of 15 percentage points (not shown in table). 

Conclusions: ASAP is one of the most ambitious efforts in the country to improve the success 
rates of low-income postsecondary students. Services at community colleges are often lacking — 
particularly when contrasted with services at four-year institutions — and initiatives such as 
ASAP may be what is needed to help the field see beyond the status quo and achieve the results 
desired by policymakers. Cost analyses will provide critical context for considering program 
effects. 
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Appendix B. Tables and Figures 

Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 1 

Selected Characteristics of Sample Members at Baseline, by Research Group 
Borough of Manhattan, Kingsborough, and LaGuardia Community Colleges 

Full Program Control 

Sample Group Group 


Gender (%) 

Male 

38.0 

36.1 

39.9 

Female 

62.0 

63.9 

60.1 

Age (%) 

16-18 years old 

39.0 

39.2 

38.8 

19-20 years old 

26.1 

25.3 

26.9 

21-24 years old 

17.9 

17.5 

18.4 

25 years old and up 

16.9 

18.0 

15.9 

Average age (years) 

21.5 

21.6 

21.3 

Marital status (%) 

Married 

6.1 

6.0 

6.3 

Unmarried 

78.6 

78.0 

79.1 

Missing 

15.3 

16.0 

14.6 

Lives with parents (%) 

73.7 

73.9 

73.4 

Parents pay more than half of expenses (%) 

40.9 

41.7 

40.1 

Missing 

18.1 

17.1 

19.1 

Race/Ethnicity a (%) 

Hispanic 

43.5 

44.5 

42.6 

White 

10.0 

10.4 

9.5 

Black 

34.4 

32.7 

36.0 

Asian or Pacific Islander 

7.5 

7.1 

7.9 

Other 3 

4.6 

5.3 

4.0 

Has one or more children (%) 

15.3 

17.4 

13.1 

Currently employed (%) 

31.3 

30.1 

32.5 


Among those currently employed: 

Number of hours worked per week in current job (%) 


1-10 hours 

8.1 

9.7 

6.7 

11-20 hours 

34.0 

37.1 

31.1 

21-30 hours 

31.7 

23.4 

39.3 

31-40 hours 

24.7 

29.0 

20.7 

More than 40 hours 

1.5 

0.8 

2.2 

Highest grade completed (%) 

10 th grade or lower 

7.2 

6.7 

7.8 

1 1th grade 

7.8 

8.4 

7.2 

12 th grade c 

75.9 

76.1 

75.8 

Missing 

9.0 

8.9 

9.2 


(continued) 
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Table 1 (continued) 



Full 

Program 

Control 


Sample 

Group 

Group 

Diplomas/degrees earned d (%) 




High school diploma 

73.7 

74.1 

73.3 

GED 

20.9 

20.6 

21.2 

Occupational/technical certificate 

5.6 

4.5 

6.6 

Other 

1.7 

1.6 

1.8 

None 

6.0 

5.9 

6.2 

Date of high school graduation/GED receipt (%) 




During the past year 

52.1 

53.8 

50.4 

Between one and two years ago 

14.0 

13.2 

14.7 

Between two and five years ago 

13.7 

13.2 

14.3 

More than five years ago 

13.8 

13.7 

14.0 

Has not earned a diploma/GED 1- 

6.3 

6.0 

6.7 

Student's status at the start of ASAP (%) 




Incoming freshman 

60.1 

62.7 

57.4 

Returning student 

33.4 

31.9 

35.0 

Transfer student 

6.5 

5.3 

7.6 

Highest degree planning to attain (%) 




Associate's 

2.8 

3.0 

2.6 

Bachelor's 

31.4 

32.6 

30.1 

Master's 

41.6 

41.3 

41.8 

Professional or Doctorate 

17.9 

17.4 

18.5 

Bachelor's, Master's or PhD (unspecified) 

6.4 

5.7 

7.0 

First person in family to attend college (%) 

28.9 

26.8 

30.9 

Missing 

5.1 

5.1 

5.2 

Highest degree/diploma earned by mother (%) 




Not a high school graduate 

19.5 

18.6 

20.4 

High school diploma or GED 

21.7 

21.5 

22.0 

Some college, did not complete a degree 

15.9 

16.0 

15.9 

College degree (AA, BA, MA, PhD) 

18.6 

18.4 

18.8 

Missing 

24.2 

25.5 

22.9 

Language other than English spoken regularly in home (' 

44.7 

45.3 

44.1 

Sample size 

897 

451 

446 

(continued) 
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Table 1 (continued) 


SOURCE: Authors' calculations using Baseline Information Form (BIF) data. 

NOTES: To analyze whether baseline characteristics jointly predicted research group status, a likelihood 
ratio test was performed. This yielded a p-value of 0.91. Convention suggests that these probabilities are large 
enough that these potential differences can be ignored in the analyses. 

A two-tailed t-test was applied to differences between the program group and control group for variables 
that are not mutually exclusive and mutually exhaustive (e.g., diplomas/degrees earned). A chi-squared test 
was applied to differences between the groups of categorical variables that are mutually exclusive and 
mutually exhaustive (e.g., race/ethnicity). Levels for statistically significant differences between program and 
control groups are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. 

Italic type indicates nonexperimental data. Significance tests are not calculated for nonexperimental data. 
Missing values are only included in variable distributions for characteristics with more than 5 percent of 
the sample missing. 

Distributions may not add to 100 percent because of rounding. 

“Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. 
Respondents who said they are not Hispanic and chose more than one race are included in the Other category. 
b Other includes Multi-racial, Native American/Alaskan Native, and other race/ethnicities. 

“This includes students who were currently enrolled in high school at study intake. 
d Distributions may not add to 100 percent because categories are not mutually exclusive. 
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Figure 1: Enrollment Rates over Two Years 
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SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** p < .01; ** p < .05; * p < .10. 

Estimates are adjusted by college and research cohort. 

Enrollment is based on courses that students are still enrolled in as of the end of the add/drop period. 
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Figure 2: Credit Accumulation over Two Years 
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SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** p < .01; ** p < .05; * p < .10. 

Estimates are adjusted by college and research cohort. 
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Table 2: Graduation Rates over Two-Years 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

P-value 

Earned a degree from any college 3 

Semester 1 

0.0 

0.0 

0.0 


Semester 2 

0.2 

0.0 

0.2 

0.318 

Semester 3 

2.9 

1.1 

1.7 * 

0.064 

Semester 4 

14.5 

8.7 

5 7 *** 

0.007 

Highest degree earned 

Certificate 

0.0 

0.0 

0.0 


Associate's degree 

14.5 

8.7 

5 y *** 

0.007 

Bachelor's degree or higher 

0.0 

0.0 

0.0 


Sample size (total = 896) 

451 

445 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are indicated 
as: *** p < .01; ** p < .05; * p < .10. 

Estimates are adjusted by college and cohort. 

“Degree receipt is cumulative - those who earned a degree in an earlier semester are counted as having a degree in 
subsequent semesters. 
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